Unless otherwise indicated herein, contents set forth in this section are neither a prior art to the claims of the present application, nor construed as a prior art despite the inclusion in this section. Augmented Reality (AR) is a technology that overlaps a 3D virtual image on a real image or background, and displays it as a single picture. As an example of the use of the augmented reality technology, an augmented reality fighting game is a form in which ‘I in reality’ confront a virtual enemy in ‘a real space.’ The augmented reality is characterized by being more realistic than a virtual reality because it overlaps a virtual image in the reality space. In addition, the virtual reality is generally used only in special environments such as a film and video field, but the augmented reality is currently popular enough to be widely used by the general public. Specifically, a map search, a location search, and the like through the internet are included in the augmented reality in a broad sense. However, since computers are difficult to use while moving, attention has been paid to augmented reality technologies targeting portable smart terminals such as smart phones and tablet PCs.
However, the conventional augmented reality technology has problems that the controller operating a task, such as the user's gesture or hand gesture, interferes with a view. There is mismatch between a screen the user views, i.e., a view by the user's natural sight, and a task screen which is displayed after being shot by a camera, thereby making it difficult to provide intuitive feedback and requiring a lot of time to training in performing a task.
In addition, even when shooting with a camera equipped in the smart terminal, a screen and an actual visible background are distorted and displayed on a front display of the smart terminal due to mismatches in a distance and an angle between a rear camera lens of the smart terminal and a pupil. Thus, it is difficult to accurately reflect a desired screen and picture when outputting a screen or taking photos or videos in the augmented reality. Specifically, referring to FIG. 1A which illustrates a conventional augmented reality display, upon comparison of an image 20 of a subject as viewed by the user's natural sight 10 and an image 50 shot by the rear camera of the smart terminal and displayed on a front of the smart terminal, a focus, a size, and a magnification, etc. of the subject included in the image displayed on the front of the smart terminal are different from those of the natural sight actually seen by the user and are distorted by shooting conditions of the rear camera 40. Since the image 50 different from an image viewed by an actual view is displayed on the front of the smart terminal, when using augmented reality related applications (for example, Pokemon GO 60, etc.), there is a mismatch in an angle and distance between a scene viewed with a naked eye and a screen of a device, thereby often reducing the liveliness. In addition, in a general camera shooting, when the user intends to shoot with his/her natural sight conditions, it is cumbersome because the user has to adjust the shooting conditions himself/herself to be the same as the natural sight conditions. Further, in practice, since it is very difficult to control the shooting conditions to be the same as the natural sight conditions, it is not easy to obtain the same image as the natural sight in case of shooting pictures or videos.